User interface for a three-dimensional colour ultrasound imaging system

ABSTRACT

A three-dimensional ultrasound imaging system color user interface generates a volumetrically-rendered ultrasound image. The ultrasound image is manipulable using three-dimensional image compositing functions. The interface presents to the user three-dimensional image controls for controlling the ultrasound image. The three-dimensional image controls have an operational similarity to two-dimensional image controls for controlling a two-dimensional ultrasound image. The interface further relates the three-dimensional ultrasound image controls to the plurality of three-dimensional image compositing functions for manipulating the ultrasound images. Additionally, the interface presents to the user a three-dimensional color image control for controllably manipulating the ultrasound image using three-dimensional image using compositing functions such as a color flow mapping function, a color flow overlay function for mapping fluid flow direction, a depth-based velocity visualization color mapping function; and an absolute velocity representation function for mapping absolute fluid flow velocity relating to said ultrasound object.

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. provisional application Ser.No. 60/477,206 filed Jun. 10, 2003, which is incorporated herein byreference.

The present invention relates to ultrasound imaging systems and theirmethods of operation and, in particular, to a three-dimensionalultrasound imaging system color user interface.

Diagnostic ultrasound equipment transmits sound energy into the humanbody and receives signals reflecting off tissue and organs such as theheart, liver, kidney, etc. Blood flow patterns are obtained from Dopplershifts or shifts in time domain cross correlation functions due to bloodcell motion. These produce reflected sound waves and generally may bedisplayed in a two-dimensional format known as color flow imaging orcolor velocity imaging. The ultrasound imaging system emits pulses overa plurality of paths and converts echoes received from objects on theplurality of paths into electrical signals used to generate ultrasounddata from which an ultrasound image can be displayed. The process ofobtaining the raw data from which the ultrasound data is produced istypically termed “scanning,” “sweeping,” or “steering a beam”.Generally, the amplitudes of reflected components for structures, suchas the heart or vessel walls, have lower absolute velocities and are 20dB to 40 dB (10-100 times) larger than the amplitudes of reflectedcomponents for blood cells.

Real-time sonography refers to the presentation of ultrasound images ina rapid sequential format as the scanning occurs. Scanning is eitherperformed mechanically (by physically oscillating one or more transducerelements) or electronically. By far, the most common type of scanning inmodem ultrasound systems is electronic scanning wherein a group oftransducer elements (termed an “array”) arranged in a line are excitedby a set of electrical pulses, one pulse per element, timed to constructa sweeping action.

Many presently used three-dimensional ultrasound imaging methods andsystems use an ultrasound probe to acquire a series of two-dimensionalplanes of data or tomographic data slices through the human body. Theassociated imaging electrodes tag these tomographic data slices withpositional information related to the two-dimensional probe. Thesetagged slices are acquired using one of a variety of acquiringtechniques, such as using a video grabber. The video grabber, forexample, takes the ultrasound image from the display on thetwo-dimensional ultrasound machine. These images may then be sent to anoff-line device for subsequent volume reconstruction.

Problems in fully using these and other ultrasound imaging techniquesrelate to the difficulties of using time gating techniques for acquiringimages of repetitive moving structures in the body, especially of theheart. In particular, there are both operational and programmingcomplexities associated with porting these two-dimensional data slicesto three-dimensional offline programs. The time needed to manipulate andto visualize this data, particularly when using off-line computerprograms, is excessive. Moreover, the data is oftentimes difficult touse and interpret.

Even for those with the skills to operate three-dimensional ultrasoundimaging systems, an important problem still exists with the datamanipulation controls of known ultrasound imaging systems. For example,heretofore the controls for color three-dimensional imaging systems areunique to three-dimensional systems. Not only are these controlsdifficult to master, but also they are non-intuitive to those trained inscanning and evaluating two-dimensional ultrasound images. This isparticularly true for those systems displaying color Dopplerinformation. Due to the complexities in understanding and using thethree-dimensional data, as well as those of using the associated volumerendering programs, only a small number of clinicians and researchershave mastered three-dimensional ultrasound systems. This hassignificantly limited the practical use and associated benefits ofthree-dimensional volume-rendering ultrasound imaging systems.

At the technical level, certain functional limitations also exist in theuse of three-dimensional volumetrically rendered ultrasound data. Forexample, known interfaces generally display color Doppler (e.g., bloodvelocity information) in conjunction with the anatomical black and white(BW) information. This results in an interaction between the BW andcolor controls. In essence, this creates a situation where a BW voxeland a color voxel, both relating to the same space-time location of anultrasound object, such as a human heart, would compete against oneanother to be visualized by a user.

Accordingly, there is a need for improved quantity in the systemcontrols for visualizing three-dimensional volumetrically-rendered colorultrasound imaging data.

A further need exists to address the problem that knownthree-dimensional ultrasound imaging systems fail to present to a user,particularly a user skilled in sonography, neither highly intuitivecontrols nor readily understandable responses to these controls.

Still further, there is a need for reducing the potentially adverseeffects of interaction between system controls when producing andanalyzing three-dimensional volumetrically rendered ultrasound data Suchinteractions may occur, for example, in the generation and use of BW andcolor data.

In accordance with the present invention, an improved three-dimensionalultrasound imaging system color user interface for using a real-time,three-dimensional, volume rendering ultrasound imaging system isprovided that substantially eliminates or reduces the disadvantages andproblems associated with prior ultrasound image system display andinterface systems.

According to one aspect of the present invention, there is provided amethod for interfacing a three-dimensional ultrasound imaging systemwith a user. The method includes the steps of generating athree-dimensional volumetrically-rendered ultrasound image of anultrasound object. The three-dimensional volumetrically-renderedultrasound image is manipulable using a plurality of three-dimensionalimage compositing functions. The method further presents to the user aplurality of three-dimensional image controls for controlling thethree-dimensional volumetrically-rendered ultrasound image. Thethree-dimensional image controls have an operational similarity totwo-dimensional image controls for controlling a two-dimensionalultrasound image. Further, the method includes the step of relating theplurality of three-dimensional ultrasound image controls to theplurality of three-dimensional image compositing functions formanipulating the three-dimensional volumetrically-rendered ultrasoundimage of the ultrasound object.

An additional feature of this aspect of the invention includes the stepsof first presenting a plurality of three-dimensional image responsesfrom operation of said three-dimensional ultrasound image controls. Theimage responses are similar to image responses of a two-dimensionalultrasound imaging system. This feature further includes the step ofcontrolling the three-dimensional volumetrically-rendered ultrasoundimage using the three-dimensional compositing functions.

According to another aspect of the invention, there is provided here amethod for interfacing a three-dimensional ultrasound imaging systemwith a user that begins with generating a three-dimensionalvolumetrically-rendered ultrasound image of fluid flow relating to anultrasound object. The three-dimensional volumetrically-renderedultrasound image is manipulable using a plurality of three-dimensionalimage compositing functions. The method presents to the user athree-dimensional color image control for controllably manipulating thethree-dimensional volumetrically-rendered ultrasound image using theplurality of three-dimensional image compositing functions. Thethree-dimensional image compositing functions are selected from a groupof such functions. The group of compositing functions includesessentially (1) a color flow mapping function for mapping fluid flowrelating to said ultrasound object; (2) a color flow overlay functionfor mapping fluid flow direction relating to said ultrasound object; (3)a depth-based velocity visualization color mapping function for mappingfluid flow depth relating to said ultrasound object; and (4) an absolutevelocity representation function for mapping absolute fluid flowvelocity relating to said ultrasound object.

A variety of further features relate to this aspect of the invention.For example, one further feature includes the step of presenting to theuser the color flow overlay function for mapping fluid flow directionrelating to said ultrasound object as an overlay function of overlayingforward and reverse flows. This feature permits generating in real-timecolor combinations arising from the overlaying function. Another furtherfeature includes the step of presenting to the user the depth-basedvelocity visualization color mapping function using a light color forshowing physically closer velocities relative to a predeterminedobservation point and a dark color associates with physically farthervelocity away from said observation point. A still further feature ofthis aspect includes presenting to the user the absolute velocityrepresentation function using absolute velocity to represent thestructure, size and position of flow pathologies associated with saidultrasound object.

A technical advantage of the present invention relates to the makingmuch easier the use of two-dimensional color Doppler controls for theoperation of and analysis supported by three-dimensional ultrasoundimaging systems. The present invention provides effective ways tocontrol complex three-dimensional compositing functions so that theirbehavior is quite similar to how such a user has generally operated andgenerated analytical information in two-dimensional color imagingsystems.

One more technical advantage of the present invention is a reduction ofthe adverse effects of interaction between system controls whenproducing and analyzing three-dimensional volumetrically renderedultrasound data. The present invention provides the ability toselectively suppress or eliminate the interaction between BW and colordata. The method and system of the present invention provide a userinterface that contemplates and effectively handles these potentiallyadverse technical interactions.

A further technical advantage of the present invention is that ofproviding a system in which there is combined into a single control aset of three-dimensional complex functions and capabilities forsubstantially enhancing the use and visualization of ultrasoundmeasurements. In a preferred embodiment, this control is referred to as“C Vision,” and contains four separate three-dimensional ultrasoundimaging modes. For example, the present invention may be viewed asproviding a “fly-by-wire” capability whereby the user, through a singleand simple interface, may affect numerous complex parameters andalgorithms that generate three-dimensional volumetrically-renderedultrasound images. Particularly important to this technical advantage isthat the present invention conceals from the user the inherentcomplexities of achieving these multiple image compositing functions. Asa result, those trained in the use of two-dimensional ultrasoundsystems, as well as new users, will find the benefits of real-time,three-dimensional, volumetrically-rendered ultrasound images much morereadily accessible.

Other technical advantages of the present invention are readily apparentto one skilled in the art from the following figures, description, andclaims.

For a more complete understanding of the present invention andadvantages thereof, reference is now made to the following descriptionwhich is to be taken in conjunction with the accompanying drawings inwhich like reference numbers indicate like features and wherein:

FIG. 1 is a diagram illustrating the use of an ultrasound diagnosticsystem that may use the present invention;

FIG. 2 is a block diagram of an ultrasound system in accordance with thepreferred embodiment of the present invention;

FIG. 3 shows conceptually the process of the present invention,beginning with ultrasound propagation and continuing through to displayof a volumetric ultrasound image on a computer monitor;

FIG. 4 portrays a view of a three-dimensional volumetric renderingultrasound imaging system console according to the teachings of thepresent invention;

FIG. 5 presents the control panel for the ultrasound imaging system ofFIG. 4;

FIG. 6 shows the control touch panels for implementing the presentinvention using an ultrasound imaging system;

FIGS. 7, 8 and 9 depict touch screen arrangements for achieving variousaspects of the present invention;

FIG. 10 depicts the numerous complex controls that are hidden and madetransparent to the user through the use of the present invention; and

FIGS. 11 through 15 provide monitor displays for demonstrating thevarious functions and capabilities of the present invention.

The preferred embodiment of the present invention and its advantages arebest understood by referring to FIGS. 1 through 15 of the drawings, likenumerals being used for like and corresponding parts of the variousdrawings.

FIG. 1 shows a simplified block diagram of an ultrasound imaging system10 that may use the concepts presented in accordance with the preferredembodiment of the present invention. It will be appreciated by those ofordinary skill in the relevant arts that ultrasound imaging system 10,as illustrated in FIG. 1, and the operation thereof as describedhereinafter is intended to be generally representative of such systemsand that any particular system may differ significantly from that shownin FIG. 1, particularly in the details of construction and operation ofsuch system. As such, ultrasound imaging system 10 is to be regarded asillustrative and exemplary and not limiting as regards the inventiondescribed herein or the claims attached hereto.

In this specification the word “system” is to be understood as genericto including one element or device and to include multiple elementsand/or devices. Also, in this specification the word “mode” is to beunderstood as generic to functions of, operations performed by,operations performed on, and/or settings of the entire ultrasound systemor of any part of the ultrasound system. Therefore, the description thatfollows of the exemplary embodiments is for purposes of illustration andnot limitation.

In certain circumstances, when it is desirable that a piece of hardwarepossess certain characteristics, these characteristics are describedmore fully in the following text. The required structures for a varietyof these machines may appear in the description given below. Machineswhich may be modified in accordance with the teachings of the presentinvention include those manufactured by such companies as PHILIPSMEDICAL SYSTEMS INTERNATIONAL, GE MEDICAL SYSTEMS, and SIEMANS MEDICALSYSTEMS, as well as other manufacturers of ultrasound equipment.

In FIG. 1, Ultrasound imaging system 10 generally includes ultrasoundunit 12 and connected transducer 14. Transducer 14 includes a receiver16. Ultrasound unit 12 has integrated therein a transmitter 18 andassociated controller 20. Controller 20 provides overall control of thesystem by providing timing and control functions. As will be discussedin detail below, the control routines include a variety of routines thatmodify the operation of receiver 16 so as to produce a volumetricultrasound image as a live real-time image, a previously recorded image,or a paused or frozen image for viewing and analysis.

Ultrasound unit 12 is also provided with imaging unit 22 for controllingthe transmission and receipt of ultrasound, and image processing unit 24for producing a display on a monitor (See FIG. 2). Image processing unit24 contains routines for rendering a three-dimensional image.

During freehand imaging, a user moves transducer 14 over subject 25 in acontrolled motion. Ultrasound unit 12 combines image data produced byimaging unit 22 with location data produced by the controller 20 toproduce a matrix of data suitable for rendering onto a monitor (See FIG.2). Ultrasound imaging system 10 integrates image rendering processeswith image processing functions using general purpose processors andPC-like architectures. On the other hand, use of ASICs to perform thestitching and rendering is possible.

FIG. 2 is a block diagram of an ultrasound system 10 in accordance withthe preferred embodiment of the present invention. The ultrasoundimaging system 10 shown in FIG. 2 is configured for the use of pulsegenerator circuits, but could be equally configured for arbitrarywaveform operation. Ultrasound imaging system 10 uses a centralizedarchitecture suitable for the incorporation of standard personalcomputer (“PC”) type components and includes transducer 14 which, in aknown manner, scans an ultrasound beam, based on a signal from atransmitter 18, through an angle. Backscattered signals, i.e., echoes,are sensed by transducer 14 and fed, through receive/transmit switch 32,to signal conditioner 34 and, in turn, to beamformer 36. Transducer 14includes elements, preferably configured as an electronically steerabletwo-dimensional array. Signal conditioner 34 receives backscatteredultrasound signals and conditions those signals by amplification andforming circuitry prior to their being fed to beam former 36. Withinbeam former 36, ultrasound signals are converted to digital values andare configured into “lines” of digital data values in accordance withamplitudes of the backscattered signals from points along an azimuth ofthe ultrasound beam.

Beamformer 36 feeds digital values to application specific integratedcircuit (ASIC) 38 which incorporates the principal processing modulesrequired to convert digital values into a form more conducive to videodisplay that feeds to monitor 40. Front end data controller 42 receiveslines of digital data values from beam former 36 and buffers each line,as received, in an area of buffer 44. After accumulating a line ofdigital data values, front end data controller 42 dispatches aninterrupt signal, via bus 46, to shared central processing unit (CPU)48, which may be a MOTOROLA PowerPC. CPU 48 executes control procedures50 including procedures that are operative to enable individual,asynchronous operation of each of the processing modules within ASIC 38.More particularly, upon receiving an interrupt signal, CPU 48 feeds aline of digital data values residing in buffer 44 to random accessmemory (RAM) controller 52 for storage in random access memory (RAM) 54which constitutes a unified, shared memory. RAM 54 also storesinstructions and data for CPU 48 including lines of digital data valuesand data being transferred between individual modules in ASIC 38, allunder control of RAM controller 52.

Transducer 14, as mentioned above, incorporates receiver 16 thatoperates in connection with transmitter 18 to generate locationinformation. The location information is supplied to (or created by)controller 20 which outputs location data in a known manner. Locationdata is stored (under the control of the CPU 48) in RAM 54 inconjunction with the storage of other digital data values.

Control procedures 50 control front end timing controller 45 to outputtiming signals to transmitter 28, signal conditioner 34, beam former 36,and controller 20 so as to synchronize their operations with theoperations of modules within ASIC 38. Front end timing controller 45further issues timing signals which control the operation of the bus 46and various other functions within the ASIC 38.

As aforesaid, control procedures 50 configure CPU 48 to enable front enddata controller 42 to move the lines of digital data values and locationinformation into RAM controller 52 where they are then stored in RAM 54.Since CPU 48 controls the transfer of lines of digital data values, itsenses when an entire image frame has been stored in RAM 54. At thispoint, CPU 48 is configured by control procedures 50 and recognizes thatdata is available for operation by scan converter 58. At this point,therefore, CPU 48 notifies scan converter 58 that it can access theframe of data from RAM 54 for processing.

To access the data in RAM 54 (via RAM controller 52), scan converter 58interrupts CPU 48 to request a line of the data frame from RAM 54. Suchdata is then transferred to buffer 60 of scan converter 58 and istransformed into data that is based on an X-Y coordinate system. Whenthis data is coupled with the location data from controller 20, a matrixof data in an X-Y-Z coordinate system results. A four-(4-) dimensionalmatrix may be used for 4-D (X-Y-Z-time) data. This process is repeatedfor subsequent digital data values of the image frame from RAM 54. Theresulting processed data is returned, via RAM controller 52, into RAM 54as display data. The display data is stored separately from the dataproduced by beam former 36. CPU 48 and control procedures 50, via theinterrupt procedure described above, sense the completion of theoperation of scan converter 58. Video processor 64, such as theMITSUBISHI VOLUMEPRO series of cards, interrupts CPU 48 which respondsby feeding lines of video data from RAM 54 into buffer 62, which isassociated with the video processor 64. Video processor 64 uses videodata to render a three-dimensional volumetric ultrasound image as atwo-dimensional image on monitor 40. Further details of the processingof three dimensional cardiac data may be found in U.S. Pat. No.5,993,390.

FIG. 3 shows conceptually the ultrasound imaging process of the presentinvention, beginning with ultrasound propagation and continuing throughto the display of a volumetric ultrasound image on computer monitor 40.In the example shown in FIG. 3, there are slices 66 conjoined at singleapex 68, but otherwise separated. Each of scan lines 70 in slices 66 hasa matching (or “indexed”) scan line in the other slices. Preferably,scan lines 70 with the same lateral position are matched across the setof slices. One way to accomplish this is to index each of the scan linesin a slice by numbering them in sequence. Then scan lines 70 having thesame index value can be easily matched.

To render a volumetric three-dimensional image, data points on each ofsets of matched scan lines 70 are linearly combined using an additionroutine. In other words, each slice in the set of slices is accumulatedin the elevation direction to produce an aggregate slice for subsequentdisplay. Preferably, but not necessarily, the data points in each sliceare weighted, for example, on a line-by-line basis by using a multiplyand accumulate routine (also known as a “MAC routine”).

FIG. 3 further illustrates the processing of ultrasound data, forexample of human heart 72, using volumetric ultrasound processing forwhich the present invention has particular beneficial application. Inone embodiment, the present invention has particularly beneficial usewith a live, three-dimensional ultrasound architecture thatinstantaneously processes data from slice 66 arising from the use oftransducer 14 to produce voxel matrix 74 of data. Voxel matrix 74,through the use a powerful supercomputer architecture such as that ofthe SONOS 7500 System manufactured by Philips Medical Systems, processeswithin a small amount of time, nominally 50 milliseconds, streamingthree-dimensional ultrasound data. This processed ultrasound data maythen appear on a monitor screen 40 to show in real-time,volumetrically-rendered ultrasound object 76.

A three-dimensional system such as the SONOS 7500 with which the presentinvention operates uses transducer 14, which includes a 3000-elementarray, and associated microprocessors that preprocess data using anadvanced, yet PC-based, computing platform, as well as special softwarethat allows interactive image manipulation and an easy-to-use operatorinterface. The 3000-element array captures data about an ultrasoundobject, such as the heart, as a volume. By combining a transducercrystal that is etched to have the necessary number of crystals with amicroprocessing circuit that efficiently triggers the transducerelements, the ultrasonic imaging system with which the present inventionoperates harnesses the computing power of more than 150 computer boards.Further details of such an array and microprocessors are described inU.S. Pat. Nos. 5,997,479; 6,013,032; and 6,126,602.

The processing architecture includes both hardware and software thatallows real-time generation of volume data. This PC-based technologysupports instantaneous display of three-dimensional images. With thistechnology, the ultrasound imaging system applies the 3000 channels tothe SONOS 7500 mainframe beam former for scanning in real time.Three-dimensional scan converter 58 (FIG. 2) processes at a rate of over0.3 giga-voxels per second to produce image 76 from voxel matrix 74.

The present embodiment of the invention, therefore, may be employed in athree-dimensional live ultrasound imaging and display process to enhanceknown echocardiography analysis and diagnosis. The system with which thepresent invention may operate has the ability to generate and displaythree-dimensional images of a beating heart an instant after the dataare acquired. However, the present invention may also operation withother, near-real-time three-dimensional systems which may need severalseconds to acquire the data and additional time to reconstruct it as athree-dimensional ultrasound display. In such systems, data acquisitionleading to three-dimensional ultrasound images of the heart may be gatedfor electrocardiogram and respiration analysis and diagnosis.

The system with which the present invention preferably operates deliversa full-volume view of the heart that can be rotated to allow theoperator to see cardiac anatomy from several perspectives. Images canalso be cropped to obtain cross-sectional pictures of complex anatomicalfeatures such as heart valves. The preferred ultrasound system for usingthe present invention can also provide information about a patient'sheart size, shape, and anatomic relationships. Such a system isattractive to a wide range of medical environments from the communityhospital and university echo lab to private offices. Thethree-dimensional capability of such a system allows a better appraisalof the correlation between valves, chambers, and vessels in the heart.

The live, volumetric ultrasound system with which the present inventionpreferably operates provides improved visualization of complex anatomicfeatures, particularly in pediatrics. Typically in pediatrics,cardiologists spend quite a bit of time looking at varioustwo-dimensional planes, trying to link one part of the heart to another.Volume rendering by a system such as that of the present invention maylead to improved, faster assessment of the pediatric heart, becausephysicians can better visualize the heart and surrounding structures.

FIG. 4 portrays an isometric (isometric=?) view of a three-dimensionalvolumetric rendering ultrasound imaging system console 12 for employingthe teachings of the present invention. In FIG. 4, ultrasound imagingconsole 12 includes monitor 40 which displays the three-dimensionalvolumetrically rendered ultrasound image 76 for a user to manipulate inthe ultrasound diagnosis of patient 25. Ultrasound imaging systemconsole 12 may include optical disk drives 82, floppy disk drive 84, VCR86, touch panels 88 (including left touch panel 102 and right touchpanel 104 described below), and keyboard controls 90. Electricalconnections associated with ultrasound imaging system console 12 includetransducer connections 92, circuit breaker 94 on back side of console12, and main power switch 96. For purposes of the present invention,ultrasound imaging system console 12 provides optional peripheral slot98 and accommodates the three-dimensional volumetric imaging module 120,as herein further described. Wheel lock 101 appropriately holdsultrasound imaging system console 12 in position as the user takesultrasound measurements. The discussion that follows will lead to anexplanation of the control system, include a variety of switches,levers, and touch panel screens for controlling a system such asultrasound imaging system 10 through the interface with imaging systemconsole 12.

As FIG. 5 shows, ultrasound imaging system console 12 includes touchpanels 88 and keyboard controls 90 for control of three-dimensionalvolumetrically rendered ultrasound image 76 on monitor 40. These includeprimary touch panel 102 and secondary touch panel 104, volume controlswitch 104, and alphanumeric keyboard 106. Moreover, console 90 includesimage tuning controls 110, and measurement and trackball controls 112,and hardcopy and loop controls 114 for further manipulating ultra soundimage 76. In an embodiment, the user interface provides for randomaccess to any of the active elements through a touch screen. In theone-handed embodiment, the user can use the thumb or any other finger to“touch” and activate any of the screen elements. Since it is throughtouch panels 102 and 104 that the user accesses and uses the system andprocess of the present invention, they are further explained in thefollowing FIGUREs.

Monitor 40 displays a graphical user interface, which may have views offixed formats and/or may give the user the option to retile or rearrangethe windows in any fashion desired. The keys and/or buttons of keyboard106 may be intelligent and interactive and may change their functionaccording to the context, history, and/or state of console 12.Moreoever, in addition to keyboard 106 and touch panels 102 and 104, thepresent invention may use a microphone for voice activation of all of orany part of the control interface.

In essence, the present invention provides an interface for athree-dimensional volume rendering ultrasound imaging system thatpresents to the user the necessary hardware and software for generatinga three-dimensional volumetrically-rendered ultrasound image of anultrasound object. The three-dimensional volumetrically-renderedultrasound image, through the present invention, is manipulable using anumber of three-dimensional image compositing functions which aredescribed in greater detail below. The system and method of the presentinvention present to the user three-dimensional image controls forcontrolling the three-dimensional volumetrically-rendered ultrasoundimage that have a visual and operational similarity to two-dimensionalimage controls for controlling a two-dimensional ultrasound image. Abenefit here is that those users who are familiar with the knowntwo-dimensional systems and processes can more readily use the newerthree-dimensional ultrasound imaging systems. The present inventionfurther relates the three-dimensional ultrasound image controls to thethree-dimensional image compositing functions for manipulating thethree-dimensional volumetrically-rendered ultrasound image of theultrasound object. Moreover, the present invention presents to the userthree-dimensional image responses from operation of thethree-dimensional ultrasound image controls. The image responses aresimilar to image responses of a two-dimensional ultrasound imagingsystem and controlling the three-dimensional volumetrically-renderedultrasound images using the three-dimensional compositing functions.

In another embodiment of the invention, there is provided an interfacefor a three-dimensional ultrasound imaging system with a user. Theparticular embodiment generates a three-dimensionalvolumetrically-rendered ultrasound image of fluid flow relating to anultrasound object, said three-dimensional volumetrically-renderedultrasound image being manipulable using a plurality ofthree-dimensional image compositing functions. The method and systempresent to the user a three-dimensional color image control forcontrollably manipulating said three-dimensional volumetrically-renderedultrasound image using said plurality of three-dimensional imagecompositing functions. The three-dimensional image compositing functionsare selected from the group of such functions. The group of suchfunctions consists essentially of (1) a color flow mapping function formapping fluid flow relating to said ultrasound object; (2) a color flowoverlay function for mapping fluid flow direction relating to saidultrasound object; (3) a depth-based velocity visualization colormapping function for mapping fluid flow depth relating to saidultrasound object; and (4) an absolute velocity representation functionfor mapping absolute fluid flow velocity relating to said ultrasoundobject.

The method and system of the present invention present to the user thecolor flow overlay function for mapping fluid flow direction relating tosaid ultrasound object as an overlay function of overlaying forward andreverse flows, thereby generating in real-time color combinationsarising from said overlaying. Moreover, the invention may present to theuser the depth-based velocity visualization color mapping function formapping fluid flow depth relating to said ultrasound object. In thisfunction, a light color associates with a physically closer velocityrelative to a predetermined observation point and darker colorassociates with physically farther velocity away from said observationpoint. Still further, the present invention may present to the user saidan absolute velocity representation function for mapping absolute fluidflow velocity relating to said ultrasound object wherein absolutevelocity representation function uses absolute velocity to represent thestructure, size and position of flow pathologies associated with saidultrasound object, said absolute velocity being independent of flowdirection.

FIG. 6 shows in yet further detail primary touch panel 102, secondarytouch panel 104, and monitor 40. The display of monitor 40 includes afrequency fusion icon 103, an operational status indicator region 105,and a color bar and baseline indicator portion 107. Operational statusindicator region 105, for example, may include indications of operationssuch as the preprocessing operation, color persist characteristics, andpost processing operations. Moreover, operational status indicatorregion 105 may show the packet size and filter, as well ascharacteristics relating to the particular color map in use. Color barand baseline indicator portion 107 shows the mean highest velocitytoward and away from the transducer, as well as other informationrelating to the operation of the transducer 14.

Primary touch panel 102 contains system-specific controls. Secondarytouch panel 104 contains mode-specific controls, as well as lessfrequently used system-specific controls. System controls, such aspresets, tools, physio, and probes are located on the touch panel 102.Imaging modalities, such as 2d, mode, color, pulsed wave, continuouswave, and Angio appear on the primary right touch panel 102. Touchpanels provide controls that highlight the respective control toindicate the control is active. To turn off an active (highlighted)control, the user touches it. Additional touch and rotary controlspertaining to the selected modality also may appear on the primary righttouch panel 104.

FIG. 6 further shows rotary controls 109 that permit adjusting variouscontrols appearing in the touch panels appearing above them. Usingrotary controls 109, the present embodiment permits changing theassociated values by turning the controls to the right or to the left.These indications and associated controls are explained in greaterdetail in the on-line and otherwise available manuals for the SONOS 5500and 7500 by Philips Medical Systems, as well as similar systems by othermanufacturers. Since it is through touch panels 102 and 104, and inparticular secondary touch panel 104 that the control of the presentinvention occurs, these are described in further detail below.

FIGS. 7, 8, 9 show in further detail the aspects of the presentinvention for controlling the compositing of data associated withthree-dimensional volumetric rendering of an ultrasound object image. InFIG. 7 there appears secondary touch screen 104, which presents theprimary three-dimensional color preview controls. These include 2Dcontrol 120 and color control 122. Biplane control icon 132, colorsuppress control 136, and baseline control 138 have no effect onthree-dimensional image control, but are used for two-dimensional imagecontrol only. Baseline control unwraps aliased signals to show highervelocities flowing in one direction by lowering color assignments forvelocities flowing in the other direction. 3D color control 124 cancelsthree-dimensional color operation. Acquire control 126, when activated,proceeds with three-dimensional color acquisition using the processes ofthe present invention.

In the lower portion of the three-dimensional preview controls screen ofFIG. 7, gain control 144 affects not only two-dimensional color gain,but also three-dimensional color gain in the 3D mode of operation. Gaincontrol 144 adjusts system sensitivity to received color flow signals.Increasing the color gain percentage increases the amount of colordisplayed. Gain control 144 is particularly important inthree-dimensional image analysis, because it affects both the signal tonoise ratio and the data to noise ratio. Fusion control 146 affects theBW fusion, including three-dimensional BW fusion. Fusion optimizesfrequencies for penetration, texture or resolution. With Fusion controlchanges are reflected in the frequency fusion icon. Control of focuscontrol 140 affects the transmit focus in both two-dimensional andthree-dimensional operation, both in color, as well as BW operation.

Scale control 142 affects the pulse repetition frequency in bothtwo-dimensional and three-dimensional operation. Control of low velocitysensitivity and frame rate occurs using scale control 142. Also aliasingcontrol is achieved through the use of scale control 142. Scale control142 changes the range of color flow velocities. The user may lower thescale to see slow flow, and increase it to see higher velocities. Filtercontrol 128 affects the Doppler Wall filter, which is important forthree-dimensional operation. Filter control 128 removes low-levelsignals and reduces noise in the image. Filter modes through Filtercontrol 128 allow the user to apply various filters either during dataacquisition and/or while viewing the data. For example, filters thatsmooth the data, high pass, low pass, and notch filters may be used tofilter out noise or data that is not of interest. Focus control 140repositions the acoustic depth of the color focal zone. When adaptiveflow is on, the focus chooses the optimal color frequency.

FIG. 8 presents the secondary three-dimensional color preview controlsthrough which the user operations the system and process of the presentinvention. As with FIG. 7, the secondary three-dimensional color previewcontrols include 2D control 120 and color control 122. Secondarycontrols further include 2nd'ary Controls control 158 and ECG Triggercontrol 154. Secondary Controls control 158 allows the user to switchbetween the primary and secondary touch panels. When the control ishighlighted, the secondary controls are active. Those controls thataffect two-dimensional color only include Map Invert control 160,smoothing control 162, and Map control 165. Full cycle control 156trades off acquisition time (e.g., the number of beats of a heart)against the number of frames for the ultrasonic image. Packet controlaffects the ensemble length sensitivity versus the image frame rate.High density control 148 affects the volume size relative to theresolution and sensitivity of the data collection. Agile control 150permits selection of the RF color frequency between 2.5 and 3.2 MHz.Finally, power control 152 affects sensitivity and should be set to 0.0dB.

FIG. 9 shows the three-dimensional color controls that a user access forthe loop display function. In FIG. 9, 2D control 120 and Map Invertcontrol 160 relate to two-dimensional operation. Smoothing control 162affects only the smoothing in the three-dimensional color mode ofoperation. Loop Display control 168 turns off loop display andthree-dimensional color operation. Filter control 128 control theDoppler wall filter and permits the user to control the trade of betweenflash and low velocity operations. Color Controls control 170 allow theuser to toggle between primary and secondary in the “Cine Loop Display”mode of operation. Baseline control 138 works best in the first of the CVision control 174 modes described below.

During three-dimensional color operation, Gain control 144, whichaffects color opacity, is not as important as in the case of controllingthe three-dimensional BW gain. Color Suppress control 136 permits theuser to see only non-compromised three-dimensional BW gain.Alternatively, BW Suppress control 172 permits the user to see onlythree-dimensional color imagery without influence from BW signals. BWsuppress control 172 suppresses the black and white image that appearsoutside of the color image, thus increasing the frame rate. For furtherC Vision control, FIG. 9 shows C Vision gain control 145 and compresscontrol 147.

Color imaging includes a color mode that uses color to represent themean velocity and direction of either blood flow as a color flow ortissue as a Doppler signal. Different shades of colors in a definedcolor spectrum represent different velocities and directions of blood ortissue movement within the selected color area. Color flow is usuallyused to examine blood flow through valves or pathological orifices inthe heart, or through vessels of the body. Accordingly, C Vision control174 of the present invention provides the valuable option of selectingfrom four different color visions, each providing very different view ofa three-dimensional volumetrically rendered ultrasound image. Theseinclude in the present embodiment Color Vision 1, Color Vision 2, ColorVision 3, and Color Vision 4. Color Vision 1 uses traditional color flowmapping to represent blood flow direction and velocity. Color Vision 2provides enhanced visualization of flow direction by overlaying forwardand reverse flows. As such, in Color Vision 2 colors can be producedthat are not visible on the color bar. Color Vision 3 shows velocity anduses enhanced color mapping for better visualization of depth cues. InColor Vision 3, for example, enhanced lighter colors represent closervelocities, while darker, more saturated colors represent velocityfarther away.

Finally, Color Vision 4 is similar to the two-dimensional Power Angiofunction of the SONOS 7500 System by Philips Medical Systems. PowerAngio is an “amplitude-only” mode that translates the magnitudes ofreturning ultrasound echoes into shades of a single color. It is usedmostly with contrast imaging, because it is more sensitive to reactionsof contrast-agent micro bubbles that are struck by ultrasound.Accordingly, Color Vision 4 uses absolute velocity to represent thestructure, size and position of flow pathologies. In particular, ColorVision 4 is best used when flow direction is not critical. Thus, throughthe operation of the single C Vision control 174, the present inventionprovides a robust set of color controls that facilitate understandingand using the three-dimensional volumetrically rendered ultrasound imagedata.

In the preferred embodiment, C Vision control 174 affects the colorcompositing algorithm. For each of the four C Vision settings, thepresent invention uses four different compositing algorithms. ColorVision 1 uses a classical setting wherein the color compositing processcomposites only one scalar value per voxel, using, in one embodiment,aliased signal processing techniques, which would be familiar to oneskilled in the art of processing Doppler or color flow information.Color Vision 2 uses a process that results in a view that mimics apropane torch flame, where one can see a hotter inner blue flameencompassed within the cooler outer red flame. Color Vision 2differentiates between positive velocities that are toward transducer 14and reverse velocities that are away from transducer 14. So, each colorvoxel is described by two scalar values, a forward value or a reversevalue. These are separately composited to form a merged image in an RGBspace. Hence, blue reverse flow might overlay red forward flow toachieve an appearance similar to that of a propane flame.

Color Vision 3 provides the ability to change the hue of the velocitydepending upon the distance from the viewer to the flow velocity voxel.Like the Color Vision 2, Color Vision 3 uses two separate compositingchannels. Velocity signals close to the viewer are composited into thefirst compositing channel, whereas velocity signals far from the viewerare composited into the second compositing channel. By comparing theratio of the signals in the 2 composited channels, the display algorithmcan infer the average distance of the flow velocity signal from theviewer, and can then change the color hue on the displayed image toindicate that distance. Color Vision 4 creates an Angio-like display bycompositing the absolute values of the velocities, and treating them asone would treat an unsigned value such as BW compositing.

An important aspect of C Vision control 174 of the present inventionrelates to its compositing function. The compositing function relates tothe projection or creation of a two-dimensional image based upon theuser's three-dimensional perspective in viewing a three-dimensional datamatrix. The present invention includes a threshold function thatdetermines what voxels are to be composited. Values below the thresholdare made transparent, and will not contribute to the resultant image. Inthe present invention, the opacity relates to the weighting slope ofthose values greater than the threshold.

A high opacity (or slope) would result in a bi-stable image, whereeither voxels were either totally transparent or totally opaque. Thiswould have the effect that a user would not see any voxels “behind” anopaque voxel, as seen along a ray cast line from the viewer'sperspective. Alternatively, a low opacity, or slope, would result in asoft, ghosted like image, where all voxels above the threshold wouldprovide equal contribution to the composited ray cast, such as if theywere averaged. Such an image might resemble, for example, an x-ray.

FIG. 10 shows graphical user interface panel 176, which an applicationdeveloper of the live three-dimensional color system of the presentinvention may access, but which the general user does not access. Noticethe complexity and the plurality of controls required to controlthree-dimensional color flow. For operation of the present invention,the controls indicated with the “+” sign are collapsed into the single CVision control 174 of FIG. 9.

The present invention makes use of a color map wherein a velocity index,typically from −128 to +127, represents a velocity. With this mapping,an index may be established such that for each of a set of 256 velocityentries a unique RGB values which would result in the color seen onmonitor. In one embodiment, the present invention further provides adisplay brightness control. This provides the user with the ability tooverride the color mapping function to further enhance the perceivedbrightness of the RGB value associated with a single velocity index.

Another embodiment of the invention includes a voxel write priorityfunction that serves a key aspect of compositing. With this function,each voxel (whether color or BW) has the ability to block the lightreaching the voxels behind it from the viewer's perspective. Note thatwith volumetric rendering, each spatial voxel has both a BW anatomicalvalue and a color velocity functional value. In one simple scheme, oneof the values (either BW or color) would win the voxel. This affects theresulting ultrasound image 76. This decision is based upon each voxeltype value, and can be altered by this control. A high weighting towardcolor flow would imply that more color information would be seen at theexpense of black and white. The present invention may provide to varythis weighting as desired.

Still another aspect of the present invention includes a map writepriority and additive mapping function. As it turns out, both types ofvoxel data (color and BW) are composited to separate two-dimensionalplanes. These two planes are merged to form a single RGB image for thedisplay. One method is to simply add the RGB values. This is sometimesreferred to as additive mapping. Another approach uses selectivemapping, where only one type (BW or color) wins the pixel. The presentinvention, therefore, provides a map write priority that, in addition tothe individual pixel values (BW and Color), enables determining whichplane an individual voxel will use.

The present invention also modulates other available user controls. Forexample, depending upon the particular color compositing algorithm inuse, it may be desirable to influence other available user controls,such as wall filter and smoothing controls, in such a way so as to nothave to adjust these other controls when switching from to the differentC Vision controls 172. To accomplish this, the present inventionprovides “offsets” so that these user controls could be modulatedwithout the user knowing. These offsets, for example, may include (1) awall filter control 128 offset, (2) a smoothing control 162 offset; and(3) a color compositing gain control 145 offset.

A further aspect of the present invention includes the use oftwo-dimensional color Doppler controls for three-dimensional ultrasoundimaging. Color Doppler gain control in two-dimensional ultrasoundimaging affects the front-end gain of the Doppler echoes received fromred blood cells (or other moving structures in the body). This controlallows the user to trade off noise, which may be seen as random colorpixels against low amplitude Doppler echoes. In three-dimensionalimaging, the present invention maps color Doppler gain control to boththree-dimensional voxel opacity and to three-dimensional displaybrightness. By modifying the control, the user may affectthree-dimensional opacity, but without having to make direct adjustmentLow voxel weightings, as controlled by opacity, result in a dimmer, lessbright image. Higher gain helps to emphasize the flow, and increasedweight was given to penetrating through black and white. The presentinvention, therefore, modulates four different internal parameters(opacity, brightness, color compositing gain, and write-priority)through three-dimensional color gain control to give the illusion thatthe control acts as it would in two-dimensional color ultrasoundimaging.

Another aspect of the present invention includes the use of the WallFilter, which is a commonly used control for two-dimensional ultrasoundimaging. In two-dimensional ultrasound imaging, the filter control 128effects a high pass filter. According to the Doppler equation, lowvelocities correspond to low Doppler shifted frequencies, and highvelocities correspond to high frequencies. Therefore, a high pass filteris required for color flow imaging to eliminate the large slow movingechoes coming from the moving blood echoes. By making such a filtervariable, the user may optimally select between flash suppression(echoes from slow moving tissue) and the ability to detect motion fromthe slower velocity red blood cells. The present invention achieves theappearance and effect of the wall filter control in three-dimensionalimaging by modifying the compositing threshold. As such, with thepresent invention, lower velocity signals can be rejected (or accepted)in the volume composited image by the user who varies thethree-dimensional Filter control 128. Although this requires differentprogramming and software module interfaces, the present inventionachieves a result similar to the results perceived in two-dimensionalultrasound imaging.

Therefore, the present invention provides a number of ways to present tothe user a graphical interface providing power processes for displayingand controlling three-dimensional volumetrically rendered ultrasoundimages. The processes, however, are based, if only in perception, onprocesses with which the user may be familiar. This is due to the factthat the presentation of the display intentionally relates topresentations with which a person skilled in two-dimensional sonographymay be familiar. Accordingly, the present invention makesthree-dimensional sonography, real-time and pre-recorded, much morepractical than has previously been the case.

To illustrate these achievements, FIGS. 11 through 15 are monitordisplays 40 for the various functions and capabilities of the presentinvention. Thus, FIG. 11 shows a monitor 40 display for the output ofColor Vision 1, as described above, which shows ultrasound image 76including color area 55 as a more classical view where traditional colorflow mapping represents blood flow direction and velocity. FIGS. 12 and13 exhibit ultrasound image 76 including color area 55 the enhancedvisualization of flow direction of Color Vision 2, showing forward colorflow in FIG. 12 and reverse flow in FIG. 13. FIGS. 14A through 14Cillustrate in ultrasound image 76 including color area 55 the effect ofColor Vision 3, as herein described, wherein there appear in FIG. 14A anear velocity measurement, FIG. 14B a far velocity measurement, and FIG.14C the composite of the images from FIGS. 14A and 14B. FIG. 15 presentsultrasound image 76 including color area 55 from the here describedColor Vision 4 function, wherein absolute velocity portrays thestructure, size and position of blood flow pathologies. As is apparentfrom the fact that all of the displays in FIGS. 11 through 15 are of thesame ultrasound object, the present invention provides a wide variety ofpossible ultrasound image display potentials.

Although the invention has been described with reference to specificembodiments, it will be understood by those skilled in the art thatvarious changes may be made and equivalents may be substituted forelements thereof without departing from the true spirit and scope of theinvention. Accordingly, it is to be understood that the embodiments ofthe invention herein described are merely illustrative of theapplication of the principles of the invention. Reference herein todetails of the illustrated embodiments, therefore, is not intended tolimit the scope of the claims, which themselves recite those featuresregarded as essential to the invention.

1. A method for interfacing a three-dimensional ultrasound imagingsystem with a user, comprising the steps of: generating a multi-colorthree-dimensional volumetrically-rendered ultrasound image of anultrasound object, said multi-color three-dimensionalvolumetrically-rendered ultrasound image manipulable using a pluralityof multi-color three-dimensional image compositing functions; presentingto the user a plurality of multi-color three-dimensional image controlsfor controlling said multi-color three-dimensionalvolumetrically-rendered ultrasound image; presenting the user with aplurality of three-dimensional image responses from operation of saidthree-dimensional ultrasound image controls; automatically offsettingselected three-dimensional ultrasound image controls so as to cause saidthree-dimensional image responses to simulate responses fortwo-dimensional ultrasound image controls; and relating said pluralityof multi-color three-dimensional ultrasound image controls to controlsaid plurality of multi-color three-dimensional image compositingfunctions for manipulating said multi-color three-dimensionalvolumetrically-rendered ultrasound image of said ultrasound object. 2.The method of claim 1, further comprising the step of automaticallyoffsetting a wall filter control associated as one of said automaticallyoffset three-dimensional image controls.
 3. The method of claim 1,further comprising the step of automatically offsetting a smoothingcontrol associated as one of said automatically offset three-dimensionalimage controls.
 4. The method of claim 1, further comprising the step ofautomatically offsetting a color compositing gain control associated asone of said automatically offset three-dimensional image controls.
 5. Asystem for interfacing a three-dimensional ultrasound imaging systemwith a user, the system comprising: means for generating a multi-colorthree-dimensional volumetrically-rendered ultrasound image of anultrasound object, said multi-color three-dimensionalvolumetrically-rendered ultrasound image manipulable using a pluralityof multi-color three-dimensional image compositing functions; means forpresenting a plurality of three-dimensional image controls forcontrolling said multi-color three-dimensional volumetrically-renderedultrasound image; means for presenting the user with a plurality ofthree-dimensional image responses from operation of saidthree-dimensional ultrasound image controls; means for automaticallyoffsetting selected three-dimensional ultrasound image controls so as tocause said three-dimensional image responses to simulate responses fortwo-dimensional ultrasound image controls; and means for relating saidplurality of multi-color three-dimensional ultrasound image controls tosaid plurality of multi-color three-dimensional image compositingfunctions for manipulating said multi-color three-dimensionalvolumetrically-rendered ultrasound image of said ultrasound object. 6.The system of claim 5, further comprising instructions for automaticallyoffsetting a wall filter control associated as one of said automaticallyoffset three-dimensional image controls.
 7. The system of claim 5,further comprising instructions for automatically offsetting a smoothingcontrol associated as one of said automatically offset three-dimensionalimage controls.
 8. The system of claim 5, further comprisinginstructions for automatically offsetting a color compositing gaincontrol associated as one of said automatically offset three-dimensionalimage controls.
 9. A storage medium storing thereon processor-readableinstructions for interfacing a three-dimensional ultrasound imagingsystem with a user, the instructions causing the processor to execute amethod comprising: generating a multi-color three-dimensionalvolumetrically-rendered ultrasound image of an ultrasound object, saidmulti-color three-dimensional volumetrically-rendered ultrasound imagebeing manipulable using a plurality of multi-color three-dimensionalimage compositing functions; presenting to the user a plurality ofmulti-color three-dimensional image controls for controlling said athree-dimensional volumetrically-rendered ultrasound image; presentingthe user with a plurality of three-dimensional image responses fromoperation of said three-dimensional ultrasound image controls;automatically offsetting selected three-dimensional ultrasound imagecontrols so as to cause said three-dimensional image responses tosimulate responses for two-dimensional ultrasound image controls; andrelating said plurality of three-dimensional ultrasound image controlsto said plurality of three-dimensional image compositing functions formanipulating said three-dimensional volumetrically-rendered ultrasoundimage of said ultrasound object.
 10. The storage medium of claim 9,wherein the method further comprises storing the state of at least oneof said three-dimensional ultrasound imaging system controls at apredetermined time and in a format for subsequent recall of both thestate of the three-dimensional volumetrically-rendered ultrasound imageand the associated state of three-dimensional ultrasound imaging systemcontrols.